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ABSTRACT 



Considerable attention has been given to 



organizational effectiveness of public sector organizations at all 
levels: local, state, and federal. Public education has not escaped ' 
scrutiny. Current examinations of the effectiveness, efficiency, and 
quality of education are more intense than that of other public 
sector organizations. The organizational effectiveness of rural small 
school districts has been of particular interest. Rural schools have 
long been regarded as inferior to their counterparts in other 
settings, and in fact, rural districts have felt the pressures of 
new, potentially damaging demands on programs and services caused by 
declining enrollments, state reform initiatives, and depressed 
economic conditions. Although organizational effectiveness is a 
popular topic, no meaningful definition of the topic, nor of 
effectiveness exists. The design of the organizational effectiveness 
study described in this paper -is based on these points: (1) *-here 
cannot, at this time, be one universal model of organizational 
effectiveness because there is not a universal theory of 
organizations; and (2) it is more worthwhile to develop frameworks 
for assessing effectiveness than to try to develop theories. This 
design for assessing the organizational effectiveness of rural, small 
school districts consists of five major, mostly sequential, steps: 
(1) considering major policy and technical issues; (2) agreeing on 
how to think about the organization's various subsystems; (3) 
agreeing on what constitutes measures of quality; (4) agreeing on the 
translation of measures of quality into standards; and (5) 
establishing the presence or absence of a standard. Various steps are 
illustrated with tables. (ALL) 
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DESIGNING ORGANIZATIONAL EFFECTIVENESS 
STUDIES OF RURAL. SMALL SCHOOLS 



In recent years, policy makers as well as researchers and pract loners have 
devoted considerable attention to the question of organizational effectiveness 
of public sector organizations at all levels: local, state, and federal. The 
popularity of the topic in the policy communities can be traced to the growing 
concerns about the quality of public services* their costs, and the continuing 
need to make public sector institutions more accountable to the general 
public. The increased interest in the topic in the research and practice 
communities can be linked* in part, to the growing awareness that it is useful 
to think about organizational effectiveness as the very center of 
conceptualizations of the nature of organizations; and further* that it is the 
ultimate dependent variable in research on organizations (Cameron and Whetten* 
1983; Madaus. etc. al., 1980). 

Evidence of the growing interest in the topic can be found in Cameron and 
Whetten 's 1983 estimate that during the preceding two decades a minimum of 
seven major works have been published on the subject and several hundred 
articles and book chapters were written in that period (p. 1). The number of 
case studies with an organizational effectiveness focus completed in recent 
years would clearly increase their estimates ten-fold. 

Public education has not escaped the new wave of interest in the topic. 
It could even be argued that the current examinations of the effectiveness, 
efficiency , and quality of education are more intense than that of other 
public sector organizations in both absolute terms and in a relative sense. 
The several hundred reports produced by state commissions alone during the 
first part of the decade stands as evidence of this widespread concern about 
the effectiveness of public education. 
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The organizational effectiveness of rural small school districts in 
particular is bearing the brunt of much of the renewed interest in the quality 
of public education. Rural schools have long been regarded by many as 
inferior to their counterparts in other settings. Advocates of this assertion 
would seem to have gained support for their position in recent years as many 
rural districts have felt the pressures of new, potentially damaging demands 
on their programs and services caused by declining enrollments, state reform 
initiatives, and the depressed economic conditions facing agriculture and 
energy sectors of the economy. 

Despite the popularity of. the topic, most observers would agree with 
Cameron and tfhet ten's (1983) position that we are still without a meaningful 
definition of organizational effectiveness and, it follows, a theory of 
effectiveness (p. 1). These authors, who ha;/e produced what we regard to be 
one of the best "think pieces" on this complex topic, offer three explanations 
for the presence of multiple models of organizational effectiveness in the 
social sciences: they are products of multiple, arbitrary models of 
organizations; the construct space of organizational effectiveness is unknown; 
and, the best criteria for assessing organizational effectiveness are unknown 
(pp. 3-19), 

In their seminal work, Cameron and Whet ten offer two conclusions that have 
guided this design of organizational effectiveness studies for rural small 
school districts: (1) there cannot* at this time, be one universal model of 
organizational effectiveness (because there is not a universal theory of 
organizations); (2) it is more worthwhile to develop frameworks for assessing 
effectiveness than to try to develop theories of such. (Focusing on a limited 
set of criteria is the only way to resolve definitional and assessment 
problems. Measuring the constructs will generally lead to a better 
understanding of the constructs pp. 262-269.) 



In arguing that the engineering of effectiveness studies is more 
productive than theorizing about effectiveness* Cameron and Whetten offer this 
useful reminder: 

In this regard* organizational effectiveness is no different from other 
complex constructs in the social sciences* Constructs such as 
inte 1 1 igence * mot ivat ion * or leadership — whose construct space * by 
definition* also is not bounded — have been better understood as limited 
aspects of their total meaning have been measured. For example* a variety 
of approaches to motivation have been developed* each limited to a 
specific domain of the construct. Approaches relating to the satisfaction 
of needs. • .to increasing expectancies. • .to reinforcements. • .to task 
design. . .and so on* have each been pursued in research. These different 
approaches are not designed to replace one another* but to augment one 
another; and by pursuing these multiple models of motivation* the 
construct is understood to a greater degree (p. 267). 



Cameron and Whetten further assert: 

In assessing organizational effectiveness* a similar attack seems 
appropriate* that is* to concentrate on measuring limited domains of the 
construct. This requires making Informed choices about what criteria to 
include and what aspect of the organizational effectiveness construct 
space to focus on. . .formulating what to measure* how to measure, when to 
measure* and other practical (or engineering) concerns should receive 
first priority. These formulations should not be done haphazardly or 
thoughtlessly* however (pp. 267-268). 



RECOMMENDED DESIGN 
FOR CONDUCTING EFFECTIVENESS STUDIES 

This design for assessing the organizational effectiveness of rural, small 

school districts consists of five major, mostly sequential, steps: 

ONE: Consider a large number of complex policy and technical issues 
associated with organizational effectiveness efforts. 

TWO : Decide on the important, but largely straight-forward, 
technical issue of how best to think about the various subsystems or 
component parts of a rural small district for which measures of 
quality are to be developed. 

THREE ; Develop a consensus concerning what is to constitute measures 
of quality for each of the agreed upon subsystems. 

FOUR : Reach agreement on the translation of the measures of quality 
into operational standards for each of the subsystems that reflect 
the contextual realities of a rural small district. 

FIVE : Establish the presence or absence of a standard. 

Reaching agreement on the important but largely technical issue in 
Step Two should not prove to be difficult, nor should the final step—the 
establishment of the presence or absence of a standard. Considerable 
debate, however, is likely to surround the other three steps in the 
approach proposed here. But, this is one of the strengths of the design 
because it is through reasoned discussion that the competing interests of 
all principal stakeholders (the local governing board, staff, community, 
and the state) can be established, differing interests resolved, and the 
necessary concensus developed. 

STEP ONE: CONSIDERATION OF MAJOR POLICY 
AND TECHNICAL ISSUES 

As suggested previously, a large number of definitional, conceptual, and 
methodological Issues surround organizational effectiveness inquires. 
Thirteen of these that we regard to be central are cited in question form in 
Table One. Also, a number of options are established that should usually be 
considered for each one. £ 
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These policy and technical issues are interrelated. However, it should be 
stressed that no choice of option is inherently "right" or "wrong." it is 
also emphasized that the choice of an opcion for a particular issue must be 
internally consistent with choices made for other related issues. Failure to 
heed this caution can damage the intent of the exercise, perhaps irreversibly. 

The themes of the thirteen issues were greatly influenced by Cameron and 
Whetten (1983) as well as Brownas (1974). Hatry (1974). and Balk (1985). whose 
work we found to be particularly useful in shaping our perspective. 




6 



t 



TABLE 1 
STEP ONE 

MAJOR POLICY AND TECHNICAL ISSUES 
TO BE CONSIDERED 



Policies and Technical Issues and Major Options 



1. What is to be the theoretical approach used in the assessment? Major 
options include: a goal model approach, a systems resource model 
approach, and an integrated model approach. 

2. What is to be the purpose of the assessment? Major options include: 
suramative evaluation, formative evaluation, and a combination of these. 

3. From whose perspective/ interest is organizational effectiveness to be 
judged? Major options include: rural district board and staff, the 
public, the state education agency, and a combination or all of these. 

4. On what aspect (s) of the rural district's service delivery system is/are 
the assessment (s) to be made? Major options include: constituencies 
served, technologies employed, service provided, and a combination or all 
of these. 

5. What organizational level (s) is/are to be the focus of the assessment? 
Major options include: individual level, sub-unit level, organizational 
level, and a combination or all of these. 

6. what time frame(s) is/are to be used in the assessment? Major options 
include: short-term (two years or less), intermediate term (three to five 
years), long-term (five to ten years), and a combination or all of these. 

7. From what source(s) is/are standards to be derived on which effectiveness 
is to be judged? Major options include: local standards, state 
standards, regional accreditation standards, the research literature, and 
a combination or all of these. 

8. Who is to be responsible for the development of the standards? Major 
options include: rural district board and staff, constituent groups, 
state legislature or judicial branch, state education agency, professional 
associations, a dominant coalition and a combination or all of these. 

9. What type of data are to be used in the assessment? Major options 
include: quantitative data, qualitative data, and a combination of these. 

10. What are to be the source(s) of the data? Major options include: 
internally derived, externally derived, and a combination of these. 

11. What criteria are to be used in the selection of effectiveness measures 
and in data collection prccedures? Major options include: 
appropriateness, validity, reliability, availability, accessibility, cost 
usefulness, and a combination or all of these. 

12. Whose responsibility is it to measure the effectiveness of a rural 
district? Major options include: rural district board and staff, the 
public, the state education agency, and a combination or all of these. 

13. What use(s) is/are to be made of the results of the assessment? Major 
rnw~ options include: rural district internal use* use by the public, state 
ti\iL education use, and a combination or all of these, g 



STEP TWO: REACHING AGREEMENT 
ON HOW BEST TO THINK ABOUT THE VARIOUS SUBSYSTEMS 
ON PARTS OP THE ORGANIZATION 

The second step of the design is the important one of deciding how best to 
think about the various subsystems or parts of the organization that will best 
explain how the functions of a rural small school district are performed. The 
assumption made here is that most organizations, including educational ones, 
have some conceptual plan that drives the way they conduct their work in order 
to achieve their mission with efficiency and effectiveness. Understanding 
what this conceptual plan is and how the organization has structured Itself to 
achieve its goals is an essential prerequisite to an analysis of the 
organization. 

There are many ways to view the various subsystems or parts of a typical 
school system. For example, one could view how schools function by stressing 
how educational enterprises differ from other organizations. Four popular 
conceptualizations of this type are: (1) the perspective of schools as 
professional bureaucracies which is advanced by many contemporary writers, 

(2) Carlson's (1964) notion of the school as a domesticated organization, 

(3) the view of the school as an organized anarchy championed by Cohen and 
March (1974), and (4) Welch's (1976) perspective of the school as a loosely 
coupled system. (Gareth Morgan's [1986] work on the use of different 
metaphors as an aid to understanding the complexity of organizational life and 
organizational analysis promises to add further to the richness of the 
conceptualizations of how organizations work.) 

A mare conventional perspective Is that a rural school district, like 
other types of school districts, has a number of basic, near universal 
organizational-structural characteristics. In addition, it makes use of a 
number of processes through which the work of the organization is conducted. 
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The organizational-structural characteristics commonly considered, and which 
will be used later to illustrate several other features of this design, are: 

• governance subsystem 

• management subsystem 

• instructional program subsystem 

• instructional support services subsystem 

• client subsystem 

• staffing subsystem 

• . financial subsystem 

• facilities and equipment subsystem 

One of the most conventional, but still useful, perspectives of the major 
processes used in organizations to carry out the desired functions in each of 
the eight subsystems holds that most decision making is one of four types: 

• planning decisions - relate to the establishment of a detailed plan for 
creating* maintaining, or improving a program or activity. 

• organizing decisions - relate to the assignment of appropriate 
authority and responsibility. 

• directing decisions - relate to the successful motivation of people. 

• controlling decisions - relate to the timely identification an<? 
correction of discrepencies between the plan and actual performance. 



STEP THREE: REACHING AGREEMENT ON 
WHAT CONSTITUTES MEASURES OF QUALITY 



This step is on<* of the most difficult and complex tasks confronting the 
policy and profession communities. This is largely because the measure of 
quality (No sharp distinction between effectiveness and quality is being made 
here, even though the two terms could have different meanings depending upon 
their use in different contexts.) of a rural school district is dependent on 
the theoretical approach being used, the purpose of the assessment , the 
perspective from which effectiveness is being judged, and other important 
issues and value judgements that must be made. 
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While this step is complex, it is one of the most critical in any 
meaningful effort to judge the organizational effectiveness of a rural small 
school district , or for any type of organization. The inclusion of this 
design step prior to the development of specific standards fez each of the 
subsystems (Step Four) will go far in assuring that the standards used include 
more than the narrow indicators currently being advocated by many. Also, it 
will address the legitimate concerns of those critics of the initial round of 
school reform who contend that the use of efficiency indicators only, or 
product-oriented indicators only, will not result in sustained improvement in 
the quality of education. 

But what criteria should be used to establish measures of quality for 
assessing the organizational effectiveness of a rural small school district? 
It is possible to circumscribe a set of criteria thct will enlarge the 
relevant indicators of organizational effectiveness, avoid the endless pursuit 
of an unattainable goal, and prevent a debate for which there is no solution? 
There is a way to begin this process and Dunn's work (1981) has utility for 
our task here. 

Dunn argues for the use of six types of criteria as a way to establish the 
"explicitly stated values that underlie recommendations for actions" 
(p. 232). These are: 

• Effectiveness: Does the activity result in the achievement of a valued 
outcome? 

• Efficiency: Does the activity achieve the greatest effectiveness at 
least cost? 

• Adequacy: Does the activity, given the level of effectiveness, satisfy 
the needSt values, or opportunities that gave rise to a problem? 

• Equity: Does the activity result in effects or efforts that are fairly 
or justly distributed? 

• Responsiveness : D^*s the act ivity sat isf y the needs , preferences . or 
values of the intended groups? 
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• Appropriateness: Are the objectives and underlying assumptions of the 
measures tenable for this rural school district? (pp. 232-239) 

This design forces concentration on which criteria are to be used in 
establishing measures of quality. Consideration of all six is critical. 

The use of effectiveness criteria forces consideration of technical 
rationality in the establishment of standards. The use of efficiency criteria 
would result in the use of economic rationality. The inclusion of equity 
criteria that would assure legal and social rationality is not ignored, rhe 
use of responsiveness criteria forces consideration of the values of the 
contituency of the rural school district. Finally, as Dunn points outs, 
appropriateness refers to "the value of worth of a program's Objectives and tc 
the tenability of assumptions underlying these objectives (p. 238)." 

in this design, the six criteria would be applied to each of the eight 
subsystems or component parts of a rural small school district (or whatever 
conceptualization is judged to be useful). This can be illustrated through 
the use of a matrix, as shown in Table Two. 



TABLE 2 

WORKSHEET ILLUSTRATING APPLICATION OF CRITERIA 
FOR ESTABLISHING MEASURES OF QUALITY 



Subsystem 

Criteria A B c D E F G H 

Effectiveness 

Efficiency 

Adequacy 

Equity 

Responsiveness 
Appropriateness 



Key 

A. Governance, B. Management, C. Instructional Program, D. Instructional 
Support Services, E. Client, F. Staffing, G+ Financial, H. Facilities and 
Equipment 



The use of these six criteria for establishing measures of quality to 
judge the organizational effectiveness of rural small school districts will 
substantially improve the likelihood that the standards to be developed in the 
next step of this design are meaningful and defensable. This is a time 
consuming and difficult task which may explain why many current efforts appear 
to ignore important design issues, or tend to &ive prominence to tachnical, 
economic, or legal rationality to the exclusion of all other bases that should 
be considered in the development of good public policy. 
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STEP FOUR: REACHING AGREEMENT ON THE 
TRANSLATION OP MEASURES OF QUALITY INTO STANDARDS 

The fourth step in this design is also critical to the success of the 

endeavor. The objective here is the translation of the measure of quality 

(the focus of the preceding step) into operational standards for each of the 

eight subsyster that reflect the contextual realities of a rural small school 

district. 

There is no dearth of potentially meaningful standards for which measures 
of quality should be applied. Some of the most popular context indicators 
center on student-school and school-community characteristics. Prominent 
input indicators Include fiscal and other resource characteristics, 
instructional supports and services, instructional policies, and work force 
characteristics. Process Indicators commanding attention relate to leadership 
and organization characteristics and school climate. Student outcomes or 
product indicators continue to center on achievement, participation, and 
post-secondary schooling and employment characteristics. Frequently used 
examples of specific indicators in each of these broad categories are provided 
in Table Three. The illustrations cited are drawn from the indicators 
currently under consideration by the Maryland Governor's Commission on School 
Performance. 

The real challenge in this step is to assure that the indicators reflect 
the contextual realities of a rural small school district. This will 
ordinarily require adjustments in many of the indicators—adjustments that 
acknowledge the demonstrable differences between education in a rural small 
school district and that which occurs in an urban or suburban district. 

What is the nature of these differences? Two of the more obvious relate 
to the reduced scale operation of many rural small districts and their 
isolation. Consideration of the smaller enrollments of rural schools, a 
factor that is beyond the control of the school, requires that adjustments be 

H 
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made in the use of any proposed input indicators that focus on expenditure 
patterns of the district. Similarly* adjustments in any proposed process 
indicators that focus on school use of community resources to enrich the 
instructional program would be required to acknowledge the absence, in more 
isolated areas, of many of the conventional opportunities of this type. The 
use of context and process variables reflect the work of Stuff lebeam (1971). 
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TABLE 3 

ILLUSTRATIVE EXAMPLES OF COMMONLY 
USBi> INDICATORS OF QUALITY 



Category 



Illustrative Examples 



Context Variables 

Student/schooi characteristics 



Schcol/coramunity characteristics 



Inputs Variables 

Fiscal and other resources 



Instructional supports and services 



Instructional policy 



work force 



Process Variable 

Leadership and organization 



Curriculum and instruction 



Climate 



Product Variables 
Achievement 

Participation 



Post secondary 



Mobility , non- or limited English 
proficient , economical ly and 

educationally disadvantaged, 
handicapped , educational level 

Income level r unemployment rate , teen 
birth rate 



Financial support , financial capacity, 
expenditure patterns, financial 
effort t staffing patterns 

Staff development program, technology 
available , information management 
system* pupil personnel services* 
school psychological/pupil 
personnel/health services 

Time r program availability , course 
offerings, graduation requirements 

Educational background, certification, 
experience, salary and benefits, 
assignment , performance appraisal , 
attendance, staff turnover 



School goals and mission, opportunity 
for input , monitoring system, 
recognition and reward system 

System for evaluation of student 

progress, goal-oriented curriculum, 
defined curriculum* maximized learning 
time 

Parental and community involvement , 

academic emphasis , safe and order ly 
environment 



Student performance 

Course and program enrollment, dropout 
rate* attendance rate 

Enrollment in post secondary, 
military* employment 
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Once one moves beyond the more obvious adjustments of the type illustrated 
here, the task becomes infinitely more difficult. However , it is essential 
that the contextual realities of a rural small district be reflected in the 
development of standards. Complicating the task further is the fact that the 
contextual realities of rural districts themselves probably differ as much, if 
not more, than the differences between rural and urban, and rural and suburban 
districts. What is being argued here is an extension of the powerful thesis 
advanced by Tyack (1974), Sher (1976), Nachtigal (1982), DeYoung (1986), and 
others who have cautioned against the continuation of the "one best way 
mentality H that seems to have driven education policy in the nation for much 
of recent history to the detriment of rural schools. The authors deliberately 
extend this caution to the design of organizational effectiveness studies. 

An illustration of how one of the potential criteria available for use in 
translating measures of quality into operational standards is provided in 
Table Four. In this example, the criterion responsiV2ness is used. 
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TABLB 4 

AN ILLUSTRATION OF OPERATIONAL STANDARDS 
FOR THE CRITERION OF RESPONSIVENESS 



Subsystem 



Example 



Opportunities exist for involving the public 
in decision making so that their interests are 
represented. 

Opportunities exist for participatory decision 
making at all levels of the organization. 

Opportunities for addressing the educational 
needs of ail handicapped children are provided. 

Instructional Support Services Use is made of volunteers and other community 

resources to enrich the instructional program- 



Governance 
Management 

Instructional Program 



Client 



Staffing 



Financial 



Facilities and Equipment 



Use is made of management information system 
technologies to secure the views of the 
various constituencies that make up the client 
subsystem. 

Professional development opportunities exist 
that address the needs of the individual 
school and the school district. 

Decisions concerning resource allocations are 
consistent with the educational priorities of 
the individual school and the school district. 

Opportunities exist for the appropriate use of 
school district facilities by community groups. 
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STEP FIVE: ESTABLISHING THE PRESENCE 
OR ABSENCE OF A STANDARD 

The culminating step of this design is straightforward; the presence or 
absence of a standard is determined. The decision of what organization should 
conduct this assessment was made in step One. Issue 12 (Table one) posed the 
question of responsibility for measuring the effectiveness of a rural 
district—the rural district board and staff, the public, the state education 
agency, a combination or all of the above. 



SUMMARY 

Interest in organizational effectiveness is likely to continue to 
accelerate in the years ahead. The effectiveness of rural districts in 
particular will be continuously questioned. This design forces the initial 
consideration of, and the reaching of a concensus for, a number of major 
policy and technical issues that should guide the effort and give it an 
interna), coherence frequently lacking in effectiveness exercises. The design 
also requires that those who have responsibility for planning a study identify 
the criteria that are to be used in establishing measures of quality prior to 
the selection of indicators of quality. This important prerequisite to good 
effectiveness studies appears to be often neglected. Another important 
feature of the design is the requirement that agreement be reached on the 
contextual realities of a rural district and that these be reflected in the 
indicators of quality. 

It is our hope that this design will lead to the initiation of a number of 
integrated and cumulative assessments of the effectiveness of rural small 
school districts that will ultimately allow the circumscribing of the best 
criteria to use in shaping public policjj gacisions on this critical issue. 
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